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1 Long papers: recommendation and instruction: Animating an interactive conversational Q 
character for an educational game system 

Andrea Corradini, Manish Mehta, Niels-OIe Bernsen, Marcela Charfuelan 
January 2005 Proceedings of the 10th international conference on Intelligent user 
interfaces 

Full text available: ^ pdf(281.80 KB) Additional Information: full citation , abstract , references , index terms 

Within the framework of the project NICE (Natural Interactive Communication for 
Edutainment) [2], we have been developing an educational and entertaining computer 
game that allows children and teenagers to interact with a conversational character 
impersonating the fairy tale writer H.C. Andersen (HCA). The rationale behind our system is 
to make kids learn about HCA's life, fairy tales and historical period while playing and 
having fun. We report on the character's generation and realization of b ... 

Keywords: edutainment, embodied conversational agent, multimodal output, user 
interface 



Advancing interaction: Interacting with embodied agents in public environments 
Addolorata Cavalluzzi, Berardina De Carolis, Sebastiano Pizzutilo, Giovanni Cozzolongo 
May 2004 Proceedings of the working conference on Advanced visual interfaces 

Full text available: "H pdf(257.35 KB) Additional Information: full citation , abstract , references , index terms 

In this paper, we present the first results of a research aiming at developing an intelligent 
agent able to interact with users in public spaces through a touch screen or a personal 
device. The agent communication is adapted to the situation at both content and 
presentation levels, by generating an appropriate combination of verbal and non-verbal 
agent behaviours. 

Keywords: interface agents, personalization 



3 System papers: interface generation and annotation tools: A visual interface for a 
multimodal interactivity annotation tool: design issues and implementation solutions 
Mykola Kolodnytsky, Niels Ole Bernsen, Laila Dybkjaer 

May 2004 Proceedings of the working conference on Advanced visual interfaces 
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Full text available: ^ pdf( 113.00 KB) Additional Information: full citation , abstract , references , index terms 

This paper discusses the user interface design for the NITE WorkBench for Windows (NWB) 
which enables annotation and analysis of full natural interactive communicative behaviour 
between humans and between humans and systems. The system enables users to perceive 
voice and video data and control its presentation when performing multi-level, cross-level 
and cross-modality annotation, information visualisation for data coding and analysis, 
information retrieval, and data exploitation. 

Keywords: data annotation tools, data visualisation, interface design 



Web site engineering: Staging transformations for multimodal web interaction 
management 

Michael Narayan, Christopher Williams, Saverio Perugini, Naren Ramakrishnan 

May 2004 Proceedings of the 13th international conference on World Wide Web 



Full text available: ■g ) odf(2.76 MB) 



Additional Information: full citation , abstract , references , citings , index 
terms 



Multimodal interfaces are becoming increasingly ubiquitous with the advent of mobile 
devices, accessibility considerations, and novel software technologies that combine diverse 
interaction media. In addition to improving access and delivery capabilities, such interfaces 
enable flexible and personalized dialogs with websites, much like a conversation between 
humans. In this paper, we present a software framework for multimodal web interaction 
management that supports mixed-initiative dialogs betw ... 

Keywords: mixed-initiative interaction, out-of-turn interaction, partial evaluation, program 
transformations, web dialogs 



5 Creating tangible interfaces bv augmenting physical objects with multimodal language 
David R. McGee, Philip R. Cohen 

January 2001 Proceedings of the 6th international conference on Intelligent user 
interfaces 

Full text available: f B Pdf(560.34 KB) Additional Information: full citation , abstract, references , citings, index 
L — ^ terms 

Rasa is a tangible augmented reality environment that digitally enhances the existing 
paper-based command and control capability in a military command post. By observing and 
understanding the users' speech, pen, and touch-based multimodal language, Rasa 
computationally augments the physical objects on a command post map, linking these 
items to digital representations of the same-for example, linking a paper map to the world 
and Post-ita notes to military units. Herein, we give a thorough ac ... 

Keywords: augmented reality, human factors, invisible interfaces, mixed reality, 
multimodal interfaces, tangible interfaces 



Accepted Posters: Scripting embodied agents behaviour with CML: character markup B 
language 

Yasmine Arafa, Abe Mamdani 

January 2003 Proceedings of the 8th international conference on Intelligent user 
interfaces 

Full text available: " H pdf(678.22 KB) Additional Information: full citation , abstract , references 

Embodied agents present ongoing challenging agenda for research in multi-modal user 
interfaces and human-computer-interaction. Such agent metaphors will only be widely 
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applicable to online applications when there is a standardised way to map underlying 
engines with the visual presentation of the agents. This paper delineates the functions and 
specifications of a mark-up language for scripting the animation of virtual characters. The 
language is called: Character Mark-up Language (CML) and is an ... 

Keywords: CML, animated expression, automated animation scripting, embodied agents, 
interface agents, lifelike characters, mark-up languages 



7 ISIS: an adaptive, trilingual conversational system with interleaving interaction and 
delegation dialogs 

Helen Meng, P. C. Ching, Shuk Fong Chan, Yee Fong Wong, Cheong Chat Chan 

September 2004 ACM Transactions on Computer-Human Interaction (TOCHI), volume n 

Issue 3 

Full text available: *|| pdf(3.71 MB) Additional Information: full citation , abstract , references , index terms 

ISIS (Intelligent Speech for Information Systems) is a trilingual spoken dialog system 
(SDS) for the stocks domain. It handles two dialects of Chinese (Cantonese and Putonghua) 
as well as English— the predominant languages in our region. The system supports spoken 
language queries regarding stock market information and simulated personal portfolios. The 
conversational interface is augmented with a screen display that can capture mouse-clicks 
as well as textual input by typing or stylus-writing. ... 

Keywords: Human-computer spoken language interface, interaction and delegation dialogs 



8 Multimodal, multidevice and beyond: Tool-supported single authoring for device H 

independence and multimodality 

Rainer Simon, Florian Wegscheider, Konrad Tolar 

September 2005 Proceedings of the 7th international conference on Human computer 
interaction with mobile devices & services MobileHCI '05 

Full text available: *^pdf(1.13 MB) Additional Information: full citation , abstract , references , index terms 

With the growing proliferation of mobile computing devices, the vision of the web anytime, 
anywhere and on any device is rapidly becoming a reality. Technologies enabling device- 
independent presentation and new interaction modalities like voice or gesture are moving 
from research to commercially available products. As a result, developers are faced with the 
increasing challenge of providing user interfaces that match the capabilities of the different 
devices available. Within this paper we prese ... 

Keywords: device independence, mobile device, modality independence, multimodal 
interaction, single authoring, universal access, user interfaces, web authoring 



9 Interactive Editing Systems: Part II B 
Norman Meyrowitz, Andries van Dam 

September 1982 ACM Computing Surveys (CSUR), Volume 14 issue 3 

Full text available: *^pdf(9.17 MB) Additional Information: full citation , references , citings , index terms 



10 Session 3: Stylesheet transformations for interactive visualization: towards a Web3D Q 

chemistry curricula 
Nicholas F. Polys 

March 2003 Proceeding of the eighth international conference on 3D Web technology 
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Full text available: ^] pdf(574.50 KB) Additional Information: full citation , abstract , references , citings , index 

terms 

Recent Standards specifications offer important but underemployed techniques to maximize 
access-to and distribution-of information for real-time 3D visualization over the web. This 
paper describes and evaluates such techniques to transform structured data such as 
Chemical Markup Language (CML) to different forms and contexts for Web3D delivery using 
Extensible Stylesheet Transformations (XSLT), Extensible 3D (X3D), and VRML97. 
Standards design approaches offer a number of advantages: data durabi ... 

Keywords: education, information visualization, interactive 3D graphics, molecular 
chemistry 

11 Languages & Authoring for the Semnatic Web: Authoring and annotation of web pages Q 
in CREAM 

Siegfried Handschuh, Steffen Staab 

May 2002 Proceedings of the 11th international conference on World Wide Web 

Additional Information: full citation , abstract , references , ci tings , index 



Full text available: W j pdf(764.65 KB) 

^-^ terms 

Richly interlinked, machine-understandable data constitute the basis for the Semantic Web. 
We provide a framework, CREAM, that allows for creation of metadata. While the annotation 
mode of CREAM allows to create metadata for existing web pages, the authoring mode lets 
authors create metadata — almost for free — while putting together the content of a 
page. As a particularity of our framework, CREAM allows to create relational metadata, i.e. 
metadata that instantiate interrelated defini ... 

Keywords: RDF, annotation, metadata, semanticWeb 



12 Interactive Editing Systems: Part I H 
Norman Meyrowitz, Andries van Dam 

September 1982 ACM Computing Surveys (CSUR), Volume 14 issue 3 

Full text available: B ffl pdf(3.08 MB) Additional Information: full citation , citings , index terms 



13 Posters: DomoML: the definition of a standard markup for interoperability of human 
home interactions 

Francesco Furfari, Lorenzo Sommaruga, Claudia Soria, Roberto Fresco 
November 2004 Proceedings of the 2nd European Union symposium on Ambient 
intelligence EUSAI '04 

Full text available: ^ pdf(326.40 KB) Additional Information: full citation , abstract , references 

DomoML is a mark-up language aimed at the definition of interoperability standard for 
domestic resources within the NICHE project (Natural Interaction in Computerised Home 
Environment). The project focuses on human language as a means for mediating user 
interaction with the home environment, enabling a user to control, query and program 
devices. Various heterogeneous components are going to take part in this architecture, and 
DomoML is the glue which allows them to be interfaced. It is a mark ... 

Keywords: ambient interfaces, human home interaction, interoperability, multimodal 
interaction, natural user-system interaction, software architectures, standard markup, 
system integration and prototyping 
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14 Session 4: Implementation of a scripting language for VRML/X3D -based embodied 
agents 

Zhisheng Huang, Anton Eliens, Cees Visser 

March 2003 Proceeding of the eighth international conference on 3D Web technology 

Full text available: ^pdf(1.02 MB) Additional Information: full citation , abstract , references , index terms 

Embodied agents or humanoid avatars may effectively be used to communicate with human 
users. Currently there is a wide range of specification formalisms and scripting languages 
for embodied agents, many of which are of a somewhat ad hoc nature lacking clear 
semantics. In this paper, we discuss the implementation of a scripting language for 
humanoid avatars in VRML/X3D-based environments. The scripting language STEP is based 
on dynamic logic, which provides a clear semantics for complex behaviora ... 

Keywords: STEP, VRML/X3D, XSTEP, agents, distributed logic Programming, h-anim 



15 Interfaces: Authoring scenes for adaptive, interactive performances 
Patrick Gebhard, Michael Kipp, Martin Klesen, Thomas Rist 

July 2003 Proceedings of the second international joint conference on Autonomous 
agents and multiagent systems 

Full text available: ^ pdf(3.18 MB) Additional Information: full citation , abstract , references , index terms 

In this paper, we introduce a toolkit called SceneMaker for authoring scenes for adaptive, 
interactive performances. These performances are based on automatically generated and 
pre-scripted scenes which can be authored with the SceneMaker in a two-step approach: In 
step one, the scene flow is defined using cascaded finite state machines. In a second step, 
the content of each scene must be provided. This can be done either manually by using a 
simple scripting language, or by integrating scenes wh ... 

Keywords: authoring, believability, embodied agents, user adaptivity, virtual theater 



16 A comprehensive approach for the development of modular software architecture 
description languages 

Eric M. Dashofy, Andre van der Hoek, Richard N. Taylor 

April 2005 ACM Transactions on Software Engineering and Methodology (TOSEM), 

Volume 14 Issue 2 

Full text available: ^ pdf(3.51 MB) Additional Information: full citation , abstract , references , index terms 

Research over the past decade has revealed that modeling software architecture at the 
level of components and connectors is useful in a growing variety of contexts. This has led 
to the development of a plethora of notations for representing software architectures, each 
focusing on different aspects of the systems being modeled. In general, these notations 
have been developed without regard to reuse or extension. This makes the effort in 
adapting an existing notation to a new purpose commensurate ... 

Keywords: ArchStudio 3, Architecture description languages, XML, xADL 2.0 




17 Agents, interactions, mobility, and systems (AIMS): Knowledge-based conversational Q 

agents and virtual storytelling 
Paul Tarau, Elizabeth Figa 

March 2004 Proceedings of the 2004 ACM symposium on Applied computing 

Full text available: "jf ) pdf(461.87 KB) Additional Information: full citation , abstract , references 

We describe an architecture for building speech-enabled conversational agents, deployed as 
self-contained Web services, with ability to provide inference processing on very large 
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knowledge bases and its application to voice enabled chatbots in a virtual storytelling 
environment. The architecture integrates inference engines, natural language pattern 
matching components and story-specific information extraction from RDF/XML files. Our 
Web interface is dynamically generated by server side agents s ... 

Keywords: FrameNet and Open Mind-based knowledge processing, WordNet, agent 
architectures, agentbased Web services, conversational agents, logic programming, natural 
language and speech processing, virtual storytelling 

18 Markup systems and the future of scholarly text processing | 
James H. Coombs, Allen H. Renear, Steven J. DeRose 

November 1987 Communications of the ACM, volume 30 issue n 

. . i ui sen ah a Additional Information: full citation , abstract , references , citings , index 

Full text available: t% ] pdf(1.91 MB) 

tiJ ~^ terms , review 

Markup practices can affect the move toward systems that support scholars in the process 
of thinking and writing. Whereas procedural and presentational markup systems retard that 
movement, descriptive markup systems accelerate the pace by simplifying mechanical tasks 
and allowing the authors to focus their attention on the content. 

19 Posters: XISL: a language for describing multimodal interaction scenarios j 
Kouichi Katsurada, Yusaku Nakamura, Hirobumi Yamada, Tsuneo Nitta 

November 2003 Proceedings of the 5th international conference on Multimodal 
interfaces 

Full text available: B || pdf(152.59 KB) Additional Information: full citation , abstract , references , index terms 

This paper outlines the latest version of XISL (extensible Interaction Scenario Language). 
XISL is an XML-based markup language for web-based multimodal interaction systems. 
XISL enables to describe synchronization of multimodal inputs/outputs, dialog 
flow/transition, and some other descriptions required for multimodal interaction. XISL 
inherits these features from VoiceXML and SMIL. The original feature of XISL is that XISL 
has enough modality-extensibility. We present the basic XISL tags, outl ... 

Keywords: XISL, XML, modality extensibility, multimodal interaction 



20 Spoken dialogue technology: enabling the conversational user interface 
Michael F. McTear 

March 2002 ACM Computing Surveys (CSUR), Volume 34 issue l 

PIU . u. sen JW no7 an i^ov Additional Information: full citation , abstract , references , citings , index 

Full text available: "ran pdf(987.69 KB) ; 

fc^-*^ terms , review 

Spoken dialogue systems allow users to interact with computer-based applications such as 
databases and expert systems by using natural spoken language. The origins of spoken 
dialogue systems can be traced back to Artificial Intelligence research in the 1950s 
concerned with developing conversational interfaces. However, it is only within the last 
decade or so, with major advances in speech technology, that large-scale working systems 
have been developed and, in some cases, introduced into commerc ... 

Keywords: Dialogue management, human computer interaction, language generation, 
language understanding, speech recognition, speech synthesis 
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